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Abstract — X-means is a partitional clustering technique that 
iswell-known and widely used for its low computational cost. 
However, the performance of k-means algorithm tends to 
beaffected by skewed data distributions, i.e., imbalanced data. 
Theyoften produce clusters of relatively uniform sizes, even if 
input datahave varied a cluster size, which is called the “uniform 
effect.” Inthis paper, we analyze the causes of this effect and 
illustrate thatit probably occurs more in the k-means clustering 
process. As the minority class decreases in size, the “uniform 
effect” becomes evident. To prevent theeffect of the “uniform 
effect”, we revisit the well-known K-means algorithmand 
provide a general method to properly cluster imbalance 
distributed data. We present Imbalanced K-Means (IKM), a 
multi-purpose partitional clustering procedure that minimizes 
the clustering sum of squared error criterion, while imposing a 
hard sequentiality constraint in theclustering step. 

The proposed algorithm consists of a novel oversampling 
technique implemented by removing noisy and weak instances 
from both majority and minority classes and then oversampling 
only novel minority instances. We conduct experiments using 
twelve UCI datasets from various application domains using 
fivealgorithms for comparison on eight evaluation metrics. 
Experimental results show the effectiveness of the proposed 
clustering algorithm in clustering balanced and imbalanced 
data. 

Index Terms — Imbalanced data, k-meansclustering 

algorithms, oversampling, Imbalanced K-Means. 


I. Introduction 

Cluster analysis is a well-studied domain in data mining. In 
cluster analysis data is analyzed to find hidden relationships 
between each other to group a set of objects into clusters. One 
of the most popular methods in cluster analysis is k-means 
algorithm. The popularity and applicability of k-means 
algorithm in real time applications is due to its simplicity and 
high computational capability. Researchers have identified 
several factors [1] that may strongly affect the k-means 
clustering analysis including high dimensionality [2]— [4], 
sparseness of the data [5], noise and outliers in the data 
[6]-[8], scales of the data [9]— [12], types of attributes [13], 
[14], the fuzzy index m [15]— [1 8], initial cluster centers 
[19]— [24], and the number of clusters [25]— [27] . However, 
further investigation is the need of the hour to better 
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understand the efficiency of k-means algorithm with respect 
to the data distribution used for analysis. 

A good amount of research had done on the class balance data 
distribution for the performance analysis of k-means 
algorithm. For skewed-distributed data, the k-means 
algorithm tend to generate poor results as some instances of 
majority class are portioned into minority class, which makes 
clusters to have relatively uniform size instead of input data 
have varied cluster of non-uniform size. In [28] authors have 
defined this abnormal behavior of k-means clustering as the 
“uniform effect”. It is noteworthy that class imbalance is 
emerging as an important issue in cluster analysis especially 
for k-means type algorithms because many real-world 
problems, such as remote- sensing [29], pollution detection 
[30], risk management [31], fraud detection [32], and 
especially medical diagnosis [33]-[36] are of class 
imbalance.Furthermore, the rare class with the lowest number 
of instances is usually the class of interest from the point of 
view of the cluster analysis. 

Guhaet al.[37] early proposed to make use of multiple 
representative points to get the shape information of the 
“natural” clusters with nonspherical shapes [1] and achieve an 
improvement on noise robustness over the single-link 
algorithm. Liu et al. [3 8], proposed a multiprototype 
clustering algorithm, which applies thek-means algorithm to 
discover clusters of arbitrary shapes and sizes. However, 
there are following problems in the real applications of these 
algorithms to cluster imbalanced data. 1) These algorithms 
depend on a set of parameters whose tuning is problematic in 
practical cases. 2) These algorithms make use of the randomly 
sampling technique to find cluster centers. However, when 
data are imbalanced, the selected samples more probably 
come from the majority classes than the minority classes. 3) 
The number of clusters k needs to be determined in advance as 
an input to these algorithms. In a real dataset, k is usually 
unknown. 4) The separation measures between subclusters 
that are defined by these algorithms cannot effectively 
identify the complex boundary between two subclusters. 5) 
The definition of clusters in these algorithms is different from 
that of k-means. Xiongs al. [33] provided a formal and 
organized study of the effect of skewed data distributions on 
the hard k-means clustering. However, the theoretic analysis 
is only based on the hard k-means algorithm. Their 
shortcomings are analyzed and a novel algorithm is proposed. 

This paper focuses on clustering of binary dataset problems. 
The rest of this paper is organized as follows: Section 2 
presents the concept of class imbalance learning and the 
uniform effect in k-means algorithm. Section 3 presents the 
main related work about k-means clustering algorithm. 
Section 4 provides a detailed explanation of the Imbalanced 
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K-Means algorithm. Section 5 presents the datasets used for 
experiments. Section 6 presents the algorithms used for 
comparison. Section 7 presents the experimental results. 
Section 8 draws the conclusions and points out future 
research. 


II. Class Imbalance Learning 

One of the most popular techniques for alleviating the 
problems associated with class imbalance is data sampling. 
Data sampling alters the distribution of the training data to 
achieve a more balanced training data set. This can be 
accomplished in one of two ways: under sampling or 
oversampling. Under sampling removes majority class 
examples from the training data, while oversampling adds 
examples to the minority class. Both techniques can be 
performed either randomly or intelligently. 

The random sampling techniques either duplicate 
(oversampling) or remove (under sampling) random 
examples from the training data. Synthetic minority 
oversampling technique (SMOTE) [2] is a more intelligent 
oversampling technique that creates new minority class 
examples, rather than duplicating existing ones. Wilson’s 
editing (WE) [3] intelligently undersamples data by only 
removing examples that are thought to be noisy. In this study, 
we investigate the impact of intelligent oversampling 
technique on the performance of the clustering 
algorithms. While the impacts of noise and imbalance have 
been frequently investigated in isolation, their combined 
impacts have not received enough attention in research, 
particularly with respect to clustering algorithms. To alleviate 
this deficiency, we present a comprehensive empirical 
investigation of learning from noisy and imbalanced data 
using k-means clustering algorithm. 

Finding minority class examples effectively and accurately 
without losing overall performance is the objective of class 
imbalance learning. The fundamental issue to be resolved is 
that the clustering ability of most standard learning algorithms 
is significantly compromised by imbalanced class 
distributions. They often give high overall accuracy, but form 
very specific rules and exhibit poor generalization for the 
small class. In other words, overfitting happens to the 
minority class [6], [36], [37], [38], [39]. Correspondingly, the 
majority class is often over generalized. Particular attention is 
necessary for each class. It is important to know if a 
performance improvement happens to both classes and just 
one class alone. 

Many algorithms and methods have been proposed to 
ameliorate the effect of class imbalance on the performance of 
learning algorithms. There are three main approaches to these 
methods. 

• Internal approaches acting on the algorithm. These 
approaches modify the learning algorithm to deal with the 
imbalance problem. They can adapt the decision threshold to 
create a bias toward the minority class or introduce costs in 
the learning process to compensate the minority class. 


• External approaches acting on the data. These algorithms 
act on the data instead of the learning method. They have the 
advantage of being independent from the classifier used. 
There are two basic approaches: oversampling the minority 
class and undersampling the majority class. 

• Combined approaches that are based on boosting 
accounting for the imbalance in the training set. These 
methods modify the basic boosting method to account for 
minority class underrepresentation in the data set. There are 
two principal advantages of choosing sampling over 
cost-sensitive methods. First, sampling is more general as it 
does not depend on the possibility of adapting a certain 
algorithm to work with classification costs. Second, the 
learning algorithm is not modified, which can cause 
difficulties and add additional parameters to be tuned. 

III. Related Work: 

In, this section, we first review the major research about 
clustering in class imbalance learning and explain why we 
choose oversampling as our technique in this paper. Then, we 
introduce frequently used ensemble methods and evaluation 
criteria in class imbalance learning 

In recent years, clustering techniques have received much 
attention in wide areas of applicability such as medicine, 
engineering, finance and biotechnology. The main intention 
of clustering is to group data together which are having 
similar characteristics. Kaufman and Rousseeuw (1990) 
referred to clustering as “the art of finding groups in data”. It’s 
not fair to declare one clustering method as the best clustering 
method since the success of clustering method will highly 
depend on the type of data and the way of investigation for a 
specific applicability. Although many researchers attempted 
to make clustering process as a pure statistical technique but 
still largely it is regarded as an exploration procedure for 
finding the similar group of data. 

Haitaoxiang et al., [39]have proposed a localclustering 
ensemble learning method based on improved AdaBoost 
(LCEM) for rare class analysis. LCEM usesan improved 
weight updating mechanism where the weights of samples 
which are invariably correctly classified will be reduced while 
that of samples which are partially correctly classified will be 
increased. The proposed algorithm also perform clustering on 
normal class and produce sub-classes with relatively balanced 
sizes. AmuthanPrabakar et al., [40] have proposed a 
supervised network anomaly detection algorithm by the 
combination of k-means and C4.5 decision tree exclusively 
used for portioning and model building of the intrusion data. 
The proposed method is used mitigating the Forced 
Assignment and Class Dominance problems of the k-Means 
method. 

Li Xuan et al., [41]have proposed two methods, in first 
method they applied random sampling of majority subset to 
form multiple balanced datasets for clustering and in second 
methodthey observed the clustering partitions of all the 
objects in the dataset under the condition of balance and 
imbalance at a different angle. Christos Bouraset al., [42]have 
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proposed W-k meansclustering algorithm for applicability on 
a corpus of news articles derived from major news portals. 
The proposed algorithm is an enhancement of standard 
k-means algorithm using the external knowledge for enriching 
the “bag of words” used prior to the clustering process 
andassisting the label generation procedure following it. 

P.Y. Mok et al., [43]have proposed a new clustering analysis 
method that identifies the desired cluster number and 
produces, at the same time, reliable clustering solutions. It 
first obtains many clustering results from a specific algorithm, 
such as Fuzzy C-Means (FCM), and then integrates these 
different results as a judgment matrix. An iterative 
graph-partitioning process is implemented to identify the 
desired cluster number and the final result. 

Luis A. Leivaet al., [44]have proposed Warped K-Means, a 
multi-purpose partition clustering procedure that 
minimizesthe sum of squared error criterion, while imposing a 
hard sequentiality constraint in theclassification step on 
datasets embedded implicitly with sequential information. The 
proposed algorithm is also suitable for online learning data, 
since the change of number of centroids and easy updating of 
new instances for the final cluster is possible.M.F.Jianget al., 
[45] have proposed variations of k-means algorithm to 
identify outliersby clustering the data the initial phase then 
using minimum spanning tree to identify outliers for their 
removal. 

Jie Cao et al., [46] have proposed a Summation -bAsed 
Incremental Learning (SAIL) algorithm for 
Information-theoretic K-means (Info-Kmeans) aims to cluster 
high-dimensional data, such as images featured by the 
bag-of-features (BOF) model, using K-means algorithm with 
KL-divergence as the distance. Since SAIL is a greedy 
scheme it first selects an instance from data and assigns it to 
the most suitable cluster. Then the objective-function value 
and other related variables are updated immediately after the 
assignment. The process will be repeated until some stopping 
criterion is met. One of the shortcomings is to select the 
appropriate cluster for an instance. Max Mignotte[47] has 
proposed anew and simple segmentation method based on the 
K-means clustering procedure for applicability on image 
segmentation.The proposed approach overcome the problem 
of local minima, feature space without considering spatial 
constraints and uniform effect. 


IV. Framework of IKM Algorithm 

This section presents the proposed algorithm, whose main 
characteristics are depicted in the following sections. Initially, 
the main concepts and principles of k-means are presented. 
Then, the definition of our proposed IKM is introduced in 
detail. 

K-means is one of the simplest unsupervised learning 
algorithms, first proposed by Macqueen in 1967, which has 
been used by many researchers to solve some well-known 
clustering problems [48]. The technique classifies a given 
data set into a certain number of clusters (assume k clusters). 


The algorithm first randomly initializes the clusters center. 
The next step is to calculate the distance between an object 
and the centroid of each cluster. Next each point belonging to 
a given data set is associated with the nearest center. The 
cluster centers are then re-calculated. The process is repeated 
with the aim of minimizing an objective function knows as 
squared error function given by: 



( 1 ) 


Where, ( j£. — y j is the 


Euclidean distance between the data point and cluster 
center y . , Ci is the number of data points in cluster and c is 

,th 

the number of ^ cluster centers. 


The different components of our new proposed framework are 
elaborated in the following subsection. 

In the initial stage of our frame work the dataset is applied to a 
base algorithm for identifying mostly misclassified instances 
in both majority and minority classes. The instances which are 
misclassified are mostly weak instances and removing those 
instances from the majority and minority classes will not harm 
the dataset. In fact it will be helpful for improving the quality 
of the dataset in two fold; one way by removing weak 
instances from majority class will help to reduce the problem 
of class imbalance to a minor extend. Another is the removal 
of weak instances from minority class for the purpose of 
finding good instances to recursively replicate and hybridized 
for oversampling is also the part of the goal of the framework. 
The mostly misclassified instances are identified by using a 
base algorithm in this case C4.5 [49] is used. C4.5 is one of 
the best performing algorithms in the area of supervised 
learning. Our approach is classifier independent;i.e there is no 
constraint that the same classifier (in this case C4.5) has to be 
implemented for identifying mostly misclassified instances. 
The framework is introduced, and more interested researchers 
are encouraged to vary the components of the framework for 
more exploration. 

In the next phase, the datasets is partitioned into majority and 
minority subsets. As we are concentrating on over sampling, 
we will take minority data subset for further analysis to 
generate synthetic instances. 

Minority subset can be further analyzed to find the missing or 
noisy instances so that we can eliminate those. For finding 
noisy, boarder line and missing value instances for generating 
pure minority set, one of the ways is to go through a 
preprocessing process. 

The good instances remained in the minority subset are to be 
resampled i;e both replicated and hybridized instances are 
generated. The percentage of synthetic instances generated 
will range from 0 - 100 % depending upon the percentage of 
difference of majority and minority classes in the original 
dataset. The synthetic minority instances generated can have a 
percentage of instances which can be a replica of the pure 
instances and reaming percentage of instances are of the 
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hybrid type of synthetic instances generated by combing two 
or more instances from the pure minority subset. 

The oversampled minority subset and the majority subset are 
combined to form an almost balanced dataset, which is 
applied to a clustering algorithm. In this case we have used 
k-means clustering algorithm. The improvements in the 
imbalance dataset can be made into balance or almost balance 
depending upon the pure majority subset generated. The 
maximum synthetic minority instances generated are limited 
to 100% of the pure minority set formed. Our method will be 
superior to other oversampling methods since our approach 
uses the only available pure instances in the existing minority 
set for generating synthetic instances. 

Suppose that the whole training set is T, the minority classis P 
and the majority class is N , and 

P = {pi, p2 ,..., ppnum }, N = {nl,n2 ,...,nnnum } 

where/Mt/raand nnumaie the number of minority and majority 
examples. The detailed procedure of IKMis as follows. 
Mg 


ut: A set of minor class examples P, a set of major class examples N, 
jPj<jNj , and Fj, the feature set, j > 0. 

Output: Average Measure { AUC, Precision, F-Measure, TP Rate, 
TN Rate } 

External selection Phase 

Step 1: For every pi (i= 1,2,..., pnum) in the minority class P, we 
calculate its m nearest neighbors from the whole training set T. The 
number of majority examples among the m nearest neighbors is 
denoted by m' (0 < m'< m) . 

Step 2: If m/ 2 < m'<m , namely the number of pi ’s majority nearest 
neighbors is larger than the number of its minority ones, pi is 
considered to be easily misclassified and put into a set MISCLASS. 

MISSCLASS = m' 

Remove the instances m' from the minority set. 

Step 3: For every ni(i= 1,2,..., nnum) in the majority class N, we 
calculate its m nearest neighbors from the whole training set T. The 
number of majority examples among the m nearest neighbors is 
denoted by m' (0 < m'< m) . 

Step 4: If m/ 2 < m'<m , namely the number of ni ’s minority nearest 
neighbors is larger than the number of its majority ones, ni is 
considered to be easily misclassified and put into a set MISCLASS. 

MISSCLASS = m' 

Remove the instances m' from the majority set. 

Step 5: For every pi’ (i= 1,2,..., pnum’) in the minority class P, we 
calculate its m nearest neighbors from the whole training set T. The 
number of majority examples among the m nearest neighbors is 
denoted by m' (0 < m'< m). 

If m'= m, i.e. all the m nearest neighbors of pi are majority 
examples, pi’ is considered to be noise or outliers or missing values 
and are to be removed. 

Step 6: For every pi” (i= 1,2,..., pnum”) in the minority class P, we 
calculate its m nearest neighbors from the whole training set T. The 


number of majority examples among the m nearest neighbors is 
denoted by m' (0 < m'< m). 

If 0 < m'<m/ 2 , pi is a prominent example and need to be kept in 
minority set for resampling. 

Step 7 : The examples in minority set are the prominent examples of 
the minority class P, and we can see that PRcp . We set 

PR= {p'l ,p'2 ,..., p'dnum}, 0 < dnum< pnum 

Step 8: In this step, we generate s x dnum synthetic positive 
examples from the pr examples in minority set, where s is an integer 
between 1 and k. One percentage of synthetic examples generated 
are replica of pr examples and other are the hybrid of pr examples. 

Clustering Phase 

Step 1 : Select k random instances from the training data subset as 
the centroids of the clusters Cl; C2; ...Ck. 

Step 2: For each training instance X: 

a. Compute the Euclidean distance 
D(Ci,X),i = l...k 

b. Find cluster Cq that is closest to X. 

c. Assign X to Cq. 

Update the centroid of Cq. 

(The centroid of a cluster is the arithmeticmean of the instances in 
the cluster.) 

Step 3: Repeat Step 2 until the centroids of clusters Cl; C2; ...Ck 
stabilize in terms of mean- squared error criterion. 


The algorithm 1 : IKM can be explained as follows, 

The inputs to the algorithm are minority class “p” and 
majority class “n” with the number of features j. The output of 
the algorithm will be the average measures such as AUC, 
Precision, F-measure, TP rate and TN rate produced by the 
IKM method. The algorithm is mainly divided into two 
phases: External Selection Phase and Clustering Phase. In the 
External Selection phase, the imbalanced dataset is divided 
into majority, minority subclasses and noisy, outliers are 
detected and removed from both the subclasses. Then the 
consistent instances I the minority set are replicated by both 
synthetic and hybridation techniques. In the clustering phase 
the so formed datasets is applied to clustering algorithm 
K-means and evaluation metrics are measured. 


V. Datasets 

In the study, we have considered 12 binary data-sets which 
have been collected from the KEEL [50] and UCI [51] 
machine learning repository Web sites, and they are very 
varied in their degree of complexity, number of classes, 
number of attributes, number of instances, and imbalance 
ratio (the ratio of the size of the majority class to the size of 
the minority class). The number of classes’ ranges up to 2, the 
number of attributes ranges from 8 to 60, the number of 
instances ranges from 155 to 3196, and the imbalance ratio is 
up to 3.85. This way, we have different IRs: from low 
imbalance to highly imbalanced data-sets. Table 1 
summarizes the properties of the selected data-sets: for each 
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data-set, S.no, Dataset name, the number of examples (#Ex.), 
number of attributes (#Atts.), class name of each class 
(minority and majority) and the IR. This table is ordered 
according to the name of the datasets in alphabetical order. 

Table 1 Summary of benchmark imbalanced datasets 


S.no Datasets # Ex. # Atts. Class (_,+)IR 


Breast 268 9(recurrence; no -recurrence) 2.37 

Breast_w699 9(benign; malignant) 1.90 


Colic 368 22 (yes;no)1.71 

Credit-g 1000 21 (good; bad) 2.33 

Diabetes 7688 (tested-potv; tested-negtv) 1.87 

Heart-c 30314(<50,>50_1) 1.19 

Heart-h 294 14(<50,>50_1) 1.77 

8. Heart-stat 270 14 (absent, present) 1.25 

9. Hepatitis 155 19 (die; live) 3.85 

10. Ionosphere 35134 (b;g) 1.79 

11. Kr-vs-kp3196 37(won; no win) 1.09 

12. Sonar 208 60(rock ; mine ) 1.15 


We have obtained the AUC metric estimates by means of a 
10-fold cross-validation. That is, the data-set was split into ten 
folds, each one containing 10% of the patterns of the dataset. 
For each fold, the algorithm is trained with the examples 
contained in the remaining folds and then tested with the 
current fold. The data partitions used in this paper can be 
found in UCI-dataset repository [52] so that any interested 
researcher can reproduce the experimental study. 


VI. Comparison of Algorithms and experimental 
Setup 

This section describes the algorithms used in the experimental 
study and their parameter settings, which are obtained from 
the KEEL [50] and WEKA [51] software tools. Several 
clustering methods have been selected and compared to 
determine whether the proposal is competitive in different 
domains with the other approaches. Algorithms are compared 
on equal terms and without specific settings for each data 
problem. The parameters used for the experimental study in 
all clustering methods are the optimal values from the tenfold 
cross-validation, and they are now detailed. 

Table 2Experimental Settings for standard clustering algorithms 


Algorithm Parameter Value 

K-Mea 

ns distance function Euclidean max iterations 500 

Number of clusters 2 

Density cluster to wrap k-means 
minstddev 1.0E-6 
FF number of clusters 2 

EM max iterations 100 

minstddev 1.0E-6 
Number of clusters 2 

Hierarchical distance function Euclidean Number of 
clusters 2 


K-Means: K-means clustering Density: Density based clustering 
FF: Farthest First clustering EM: Expectation Maximization 
Hier: Hierarchical clusteringclustering 


VII. Experimental Results 


In this section, we carry out the empirical comparison of our 
proposed algorithm with the benchmarks. Our aim is to 
answer several questions about the proposed learning 
algorithms in the scenario of two-class imbalanced problems. 

1) In first place, we want to analyze which one of the 
approaches is able to better handle a large amount of 
imbalanced data-sets with different IR, i.e., to show which 
one is the most robust method. 

2) We also want to investigate their improvement with respect 
to classic clustering methods and to look into the 
appropriateness of their use instead of applying a unique 
preprocessing step and training a single method. That is, 
whether the trade-off between complexity increment and 
performance enhancement is justified or not. Given the 
amount of methods in the comparison, we cannot afford it 
directly. On this account, we compared the proposed 
algorithm with each and every algorithm independently. This 
methodology allows us to obtain a better insight on the results 
by identifying the strengths and limitations of our proposed 
method on every compared algorithm. 


Table 3 hmrnan' of tenfold •cross validation performance for Accuracy on all the datasets 


DatasetsK-Means Density FF EMHier IKM 


Breast 54.26±10.S4. 53.66±10.74* 65.63±9.54o 49.29±S,10. 7Q.05±1.57o 55.7S±11.S7 

Breast i95.S2±2.2696.22±2.19o 84.94*6.96* 93.75*2.79* 6}.52ifl.44* 95.36±2.19 


Colic 

60.57=11.39* 6530*10.85* 

58.67*9.91* 

66.13*7.11* 

63.05*1.13* 

68.07i9.07 

Credit-g 

55.S6±6.77* 5637*6.72* 

62.41*6.560 

60.60±533o 

70.00±0.00o 

57.08*5.99 

Diabetes 

65.42±5.87o 65.60±5.6So 

65.16±3.42o 

64.67±5.74o 

65.llifl.34o 

63.13*5.92 

Heart-c 

mmi 8094*8.020 

6S.77ilO.SSo 

80.28*7.S2o 

54.15*2.06* 

77.94±9.81 

Heart-h 

77.52*10.54* 80.04*8.68* 

66.70*12.23* 

81.01*6.51* 

63.95*1.36* 

S3.36±6.99 

Heart-stat 

7439*11.42* 7535*11.60* 

66.S5ilO.S7* 

8i.78±6.55o 

55.54*1.60* 

77.33*11.49 

Hepatitis 

71.09*12.58* 73.15*12.16* 

72.14*12.77* 

73.83*10.53* 

79.38*2.260 

7446*11.04 

Ionosphere 

70.80*6.71*73.06*635* 

62.75*6.65* 

73.08±6.47o 

64.10*135* 

71.01*8.06 

Er-vs-kp54.72i4.77. 54.19*4.93* 

53 .3 7*3. SO* 

59.9S±3.4So 

52.13ifl.65* 

55.04*4.00 

Sonar 

52.43*10.2350.12*10.40* 

50.94zS.2S* 

49.59i9.55* 

51.78*3.41* 

52.73*11.13 



Fig. 1 Test results of Accuracy on K-means, Density, FF, EM, Hier 
and IKM for Colic, Heart-h, Kr-vs-kp and Sonar Datasets. 
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libit 4 Summit of tenfold crou valid idon perfonmnct for AUC on all flit daWi Table 7 Sl] ™' d tenfold croii validation ptrfonmnce for F-oeiure on all the datasets 


D at ssetE-Mu as Density FF 

EM Hire 

EM 



Breast 

0.519=0.100# 0503=0103# 

0.574=0.099c 

0.500=0.093# 

0.499=0.007# 

0.552=0.116 

Breasu : 0.95fc0.027 * 0.966=0.021# 

0.785=0.098# 

0.951=0.022# 

0.500=0.000# 

0.953=0.022 

Colic 

0.623=0.108# 0.633=0.092# 

0570=0114# 

0.691=0.06Sc 

0.500=0.000# 

0.689=0.036 

Credit-E 

0.534=0.063 # 0.535=0.066# 

0.521=0.057# 

0.567=0.051# 

0.500=0.000# 

0.569=0.059 

Diabetes 

0.603=0.063 # 0.613=0.063# 

0.520=0.044# 

0.670=0.070c 

0.502=0.006# 

0.625=0.059 

Heart-c 

0.735=0.093 # 0.30S=0.0Slc 

0.633=0.111# 

0.304=0.07Sc 

0.500=0.000# 

0.780=0.093 

Heart-h 

0.375=0.101# 0.795=0.096# 

0.614=0.138# 

0.792=0.073# 

0.500=0.000# 

0.337=0.070 

Hsait-itat 0.1M114# 0.756=0.114* 

0.662=0.105# 

0.551=0.06Sc 

0.504=0.013# 

0.770=0.116 

Hepatitis 

0.753=0.136# 0.7Sl=0.122c 

0.670=0.163# 

O.SOfcO.lOlc 

0.500=0.000# 

0.753=0.114 

InuDspbere 

■ 0.706=0.0S0#0.743=0.079o 

0.530=0.067# 

0.771=0.05Sc 

0.500=0.000# 

0.709=0.0S0Kr- 

7s-tp0.5MM6i 0.540=0046# 

0.531=0.039# 

0.5SS=0.032c 

0.500=0.001# 

0.550=0.040 

3 dm 

0.521=0.103#0.499=0.1M# 

0.513=0.082# 

0.497=0.096# 

0.500=0.000# 

0.527=0.111 


Datasets K-Means Density 

FF 

EM 

Hire 

EM 

Breast 0.630±0.112q 0.627=0.112c 

0.755=0.03 7 c 

0569=0.082* 

0.3210.010c 

0617=0.129 

Breast w0.96fi±0.017o 0.971=0.017c 

0.393=0.045* 

0.950=0.024* 

0.792=0.003* 

0 955=0.021 

Colic 0.603=0.162* 0.662=0. 13 7 c 

0.63 3=0. 141c 

0.67S=0.0S2c 

0. 773=0.003 c 

0.6210.146 

Credit-E 0.649=0.074c 0.65M074o 

0.7310.066c 

0.700±0.052c 

0.324=0.000c 

0613=0.076 

Diabetes 0.7310.053c 0.737=0.050c 

0.7710.049c 

0.6S5=0.055c 

0.7 39=0.003 c 


0.676=0.07SHeait-c 0.793=0.09Sc 0.S24=0.0Slc 0711=0116* O.SlbO.OSlc 

0.6S10.130* 0.733=0.100 
Heart-h 0.3110.110* 0.336=0.079* 

0.745=0.129* 

0.352=0.052c 

0.730=0.010* 

0.343=0.030 

Heart-stat 0.752=0.125c 0.761=0.134c 

0.691=0.144* 

0.337=0.059c 

0.7010.037* 

0.732=0.143 

Hepatitis 0.54^0.137* 0.532=0.153* 

0.451=0.226* 

0593=0.131* 

0.000=0.000* 

0.6710.141 

I ono sphere 0.617=0.155 *0.660=0. 157 • 

0.173=0.222* 

0.711±0.059 

0.000=0.000* 

0.7110.130 

Kr-VE-Ep0.573r0.lllc 0.553=0 111* 

0.524=0.133* 

0.6S6=0.052c 

0.672=0.097c 

0563=0.109 

Sonar 0.462=0.149* 0.447=0.11* 

0414=0257* 

0.430=0.133* 

0.149=0.270* 

0512=0.136 



Table 5 Summaiy of tenfold croii validation performance for Specificity on all the datasets 


Datasets K-Maans Density FF EM Hire EM 


Breast 0.tiM112o (L43M1M. 0J7M19O- 0.51S=0.1S2. 0.000=0.000# 0.531=0.135 

Breast MmSdh Q.979±0.Q29a Q.578±0.196* 0.99fe0.012c O.OOfcO.OOO* 0.931=0.036 

Colic 0.603=0.162# 0.773=0.157# 0.506=0.336# 0.307=0.093 # 0.000=0.000# 0.316=0.162 

Credit-E 0 649=0074c 0464=0149# 0 264=0 145# 0.470=0.100# 0000=0.000# 0.559*0132 

Diabetes 0. 739=0.053 c 0.437=0.155# 0.032=0.163 # 0746=0 145-: 0.000=0.000# 

0.507=0. 116Heart-c 0.793=0.09Sc 0.7S9=0.122c 0.633=0.225# 0.317=0.107c 

0.035=0 1340# 0.743=0.119 

Heart-h O.BltO.llOc 0.775=0.193 # 0.413=0.322.# 0.723=0.127# O.OOfcO.OOO# 

0. 301=0. 105Heart-stat 0.752=0.125# 0.775=0.155# 0.601=0.220# 0.737=0.120# 

0.019=0123# 0.799=0.149 

Hepatitis 0.549=0. 15 7c 0.696=0.143# 0.757±0.154c 0.695=0.125# l.MMOQQo 0.716=0.143 

I did sphere 0.617=0. 155 #0.699=0. 110c 0.375±0.215c 0.629^0.093# 1.000=0.000c 0.637=0.153 

Kr-vs-tpO.573H0.lllc 0.497=0. 167c 0.461=0.330c 0.3310.057* 0.0210.141* 0.4510.174 

Sonar 0.462=0.149# 0.523=0.193# 0.502=0.357# 0.4710.171* 0.763=0.420c 0.535=0.136 


Fig. 1 Test results of Accuracy on K-means, Density, FF, EM, Hier 
and IKM for Breast_w, Credit-g, Diabetes, and Sonar Datasets. 

Table P Summary of tenfold ami validation performance for FT Rate on all the dataieb 


Table t Summary ol tenfold croii validation performance for Prediion on ell the datEieti 

Datasets 

K-Means Density 

FF 

EM 

Hire 

EM 

Datasets K- Means Density 

FF 

EM 

Hire 

EM 

Breast 

0.5310.199c 0.563=0. 194c 

0.627=0. 190c 

0.482=0. 132c 

1.0010.000c 

0.4610.135 

Breast 0.718*0.0900 0.7010.033c 

0.74710.061 o 

0.707=0.03 Sc 

0 702=0 014c 

0.693=0.105 

Breast_ts0.O71O.O49c 0.021=0.029* 

0.422=0. 196c 

0004=0012* 

1.0010.000c 

00610036 

Breast w0961=0024c 0.9S10.011 

0.323=0.071* 

0.993=0.007 c 

0.655=0.004* 

0.9310.032 

Colic 

0.235=0.240* 0.227=0.157* 

0.494=0.336* 

0.193=0.093* 

1.0010.000c 

0.314=0.162 

Colic 0.734=0.107* 0. 321=0.033 c 

07110120* 

0.S33=0.069c 

06310011* 

0.793=0.110 

Credit-e 

0.5210.143c 0.5310.149c 

0.7310.145c 

0.5310.100c 

1.00040.000c 

0441=0132 

Credit-E 0.727=0.052c 0.727±0.053c 

0.7 12=0.035 c 

0.7410.036c 

0.7010.000c 

0.667=0.053 

Diabetes 

0.544=0. 145 o 0.5 13=0. 155 c 

0.91 3=0. 163 c 

0.254=0.145* 

1.0010.000c 

0.493=0.116 

Diabetes 0.7210.047c 0.734=0.051 

0.66510.044c 

0.321=0.072c 

0.65210.004c 


Heart-c 

02410143 * 0211=0122* 

0.367=0.225 c 

0 133=0 107* 

0.965=0. 134c 

0252=0119 

0.62S=0.04SHeart-c 0.797=0.094c 0.331=0.036= 0.726=0.129* 0.343=0.032= 

Heart-h 

0.2310.190c 0.225=0. 193 c 

0.5S2=0.322c 

0.272=0. 127 c 

1.0010.000c 

01910105 

0.526=0.101* 0.764=0.103 





Heart-stat 0.231=0.16Sc 0.225=0.155c 

0.3910.220c 

0.213=0. 120c 

0.93 1=0. 123 c 

0.201=0.149 

Heart-h 0.367=0.0S4c O.S77±O.OS7c 

0.735=0.119* 

0. 352=0.061 c 

0.6410.014* 

0.334=0.071 

Hepatitis 

0.3 13=0. 146c 0.304=0. 143 c 

0.243=0.154* 

0.305=0. 125 c 

0.0010.000* 

0.284=0.143 

Heart-stat 0.306=0. 116c 0.303=0. 116c 

0 706=0113* 

0.3310.073c 

0.5410.06S* 

0744=0149 

I ono sphere 0.2S10. 1 10*0.301=0. 110* 

0.125=0.215* 

0.371=0.09Sc 

0.0010.000* 

0.313=0.153 

Hepatitis 0.426=0.150* 0.453=0. 156c 

0.405=0.233* 

0.457=0.136* 

0.0010.(00* 

0.613=0.153 

Sr-vs-b0.5310.18 1 o 0.503=0.167* 

0.5310.330c 

0.6710.057c 

0.9310.141c 

0.5210.174 

I dd d sphereO. 55 7^). 147 *0.5 73=0. 145 • 

0.3010.369* 

0.5310.063* 

0.0010.000* 

0.717=0.112 

Sonar 

0.4210.195 * 0.472=0. 193 c 

0.493=0.35 7 c 

0.5310.171c 

0.232=0.420* 

0465=0136 

Er-vs-lp0.561=0.041c 0.5510.042c 

0.5Sl=0.0S2c 

0.5710.022c 

0512=0 073* 

0.5410.040 







Sonar 0.493=0.133 * 0.4610.140* 

0.422=0.233* 

0.4510.103* 

0.1110.193* 

0.5310.123 








Table Id Summary of tenfold cross validation performance for Fh" Rate on all the dataiets 


Table fi Su mmary of tenfold oral validation performance for Recallon all the datasets 


Datasets 

K- Means Density 

FF 

EM 

Hire 

EM 

Breast 

0.5 7 7=0. 153c 0.5710.159c 

0.7710.137c 

0.432=0.094* 

1.0010.004c 

0.573=0.169 

Breast_iv0.976=0.0220.953=0.029* 0.992=0.033 c 0.907=0.042* 1.0010.000c 0.9710.021 

Colic 

0.541=0.231* 0.5 32=0. 195 c 

0.635=0.229c 

0.5710.100c 

1.0010.000c 

0.561=0.221 

Credit-E 

0.595=0. 103 c 0.6010.112c 

0.77 3=0. 121c 

0.664=0.03 1c 

1.0010.000c 



0.5710.11SDiabetes 0.7610.091c 0.747=0.0S9c 0.957=0.106c 0.594=0.075* 


1.0010.000c 0.742=0.122 





Heart-c 

0.793=0.124* 0.3210.104c 

• 

r-- 

■C? 

II 

& 

0.791=0.113* 

0.9510.134c 

0.311=0.121 

Heart-h 

0.7310.171* 0.315=0.126* 

0.S110.20S* 

0.3510.073* 

KMHO.OOOc 

0.372=0.106 

Heart-stat 0.724=0.157 * 0.737=0.169* 

0.723=0.230* 

0.S42=0.0S6c 

0.935=0. 122c 

0.7410.174 

Hepatitis 

0. 324=0.225 c 0.S65=0.190c 

0.533=0.319* 

0.9010.153c 

O.OOIO.OOO* 

0.7910.194 

I ono sphere 0. 702=0. 1 3 7 *0. 7 3 7=0. 195 c 

0.135=0.234* 

0.912=0.074c 

0.0010.000* 

0. 732=0. 160Kr 

vs-kp0.624d0.197c 0.532=0.199* 

0.6010.312* 

0. 847=0. 10Oc 

0.9310.141c 

0.6210.196 

Sonar 

0.471=0.201* 0471=0215* 

0.524=0.392c 

0.525=0. 194c 

0.235=0.425* 

0.5110.130 


Datasets 

i K-Means 

Density 

FF 

EM 

Hire 

EM 

Breast 

0.423=0.153* 

0421=0159* 

0.224=0.137* 

0.513=0.094= 

0.0010.004* 

0.427=0.169 

Breast_wO.O24i0.O22 

0.047=0.029c 

0.003=0.033* 

0.093=0.042= 

0.0010.000* 

0024=0021 

Colic. 

0.4510.231c 

0.413=0.195* 

0.365=0.229* 

0.424=0.100* 

0.0010.000* 

0.4310.221 

Credit-E 

0.405=0.103* 

0394=0112* 

0.222=0.121* 

0336=0031* 

0.0010.000* 

0421=0113 

Diabetes 

0.2410.091* 

0.254=0.039* 

0.043=0.106* 

0.406=0.075 c 

0.0010.000. 



0.25 3=0. 122Heart-c 0.202=0.124= 0.174=0.104* 0.267=0.187= 0.2010.113c 


0.035=0.134* 

0.1310.121 





Heart-h 

0.214^0.171= 

0.1 35=0. 126c 

0.1910.203c 

0 144=0 073 c 

0.0010.000* 



0. 123=0. 106Heart-stat 0.276=0.157c 0.263=0. 169c 0.277=0.230= 0.153=0.086* 


0.015=0.122* 

0.2610.174Hepatitis 0.176=0.225 * 0.135=0.190'* 0.417=0.319= 


0.094=0.153* 

1.0010.000c 

0.201=0.194 




I ono sphere 0.293=0. 1 3 7 c0.2 13=0. 195 • 

0.81510.284c 

0.038=0.074* 

1.0010.000c 

0.268=0.160 

Kr-v E-kp0.3 76=0. 197 * 

0413=0 199c 

0.4010.312c 

0.153=0.100* 

0.0210.141* 

03S10196 

Sonar 

0.5210.201c 

0.5210.215c 

0.476=0.392* 

0.475=0.194* 

0. 765=0.425 c 

0431=0130 
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The clustering valuations were conducted on twelve wide 
lyused datasets. Theserea 1 world multi-dimensional 
datasetsareusedtoverifytheproposedclustering method. 

Table 3, 4, 5, 6, 7, 8, 9 and 10 reports the results of Accuracy, 
AUC, Precision, Recall, F-measure, Specificity, FP Rateand 
FN Rate respectively for all the twelve datasets from UCI. A 
two-tailed corrected resampled paired t-test [46] is used in 
this paper to determine whether the results of the 
cross-validation show that there is a difference between the 
two algorithms is significant or not. Difference in accuracy is 
considered significant when the p-value is less than 0.05 
(confidence level is greater than 95%). The results in the 
tables show that IKM has given a good improvement on all 
the clustering measures. 

Two main reasons support the conclusion achieved above. 
The first one is the decrease of instances in majority subset, 
has also given its contribution for the better performance of 
our proposed IKM algorithms. The second reason, it is 
well-known that the resampling of synthetic instances in the 
minority subset is the only way in oversampling but 
conduction proper exploration-exploitation of prominent 
instances in minority subset is the key for the success of our 
algorithm. Another reason is the deletion of noisy instances by 
the interpolation mechanism of IKM. 

Finally, we can make a global analysis of results combining 
theresults offered by Tables from 3-10: 

• Our proposals, IKMis the best performing one when 
the data sets are of imbalance category. We have 
considered a complete competitive set of methods 
and an improvement of results is expected in the 
benchmark algorithms i;eK-means, Density, FF, EM 
and Hier. However, they are not able to outperform 
IKM. In this sense, the competitive edge of IKM can 
be seen. 

Considering that IKM behaves similarly or not effective than 
K-means shows the unique properties of the datasets where 
there is scope of improvement in majority subset and not in 
minority subset. Our IKM can mainly focus on improvements 
in minority subset which is not effective for some unique 
property datasets. 


Table 11 Summ ar\' of experlm ental results fo r Improved K- 
me ans 


TN Rat e IKM v ersus K-me am 
IKM v ermi B entity 
IKM versus FF 
IKM versus EM 
IKM versus Hier 


4 0 S 

5 0 4 
9 0 3 
9 0 3 
9 0 3 


FPRate IKM versus K-me am 
IKM v ersus D ensity 
IKM v ersus FF 
IKM versus EM 
IKM versus Hier 


4 0 S 

5 0 7 

3 0 9 

4 0 S 

3 0 9 


FN Rat e IKM v ersus K-me am. 
IKM v ersus D ensity 
IKM v ersus FF 
IKM versus EM 
IKM versus Hier 


5 1 6 

7 0 5 

6 0 6 

7 0 5 

9 0 3 


The summary of experimental results of IKM on all the 
compared clustering algorithms is shown in Table 11. The 
results show that proposed IKM clustering algorithm is at 
least as effective as and at times more effective than K-means, 
Density, FF, EM and Hierarchical clustering algorithms. IKM 
compared with accuracy onK-means wins on 7 dataset and 
ties on 4 datasets and loses ononly 1 dataset. The performance 
of IKM compared withDensity based clustering wins on 9 
datasetsand losses ononly 3 datasets.The performance of IKM 
comparedwith FF wins on 8 datasets andlosses on 4 datasets. 
The validation of IKM on EM wins on 6 datasets and losses 
on 6 datasets. However, performance of IKM on Hierarchical 
clustering wins on 8 datasets and losses on 4 datasets. The 
AUC, Precision, Recall, F-measure, TN Rate, FP Rate and FN 
Rate measure have shown to perform well with respect to 
IKM. 

The strengths of our model are that IKM only over-sample the 
most prominent examples recursively thereby strengthens the 
minority class. One more point to consider is our method tries 
to remove the most misclassified instances from both majority 
and minority set. Firstly, the removal of some weak instances 
from majority set will not harm the dataset; in fact it will 
reduce the root cause of our problem of class imbalance as a 
whole by reducing majority samples in a small proportion. 
Second, the removal of weak instances from the minority set 
will again help in better generation of synthetic examples of 
both same and hybrid type. 


TL e suit s Sy ste m s 

Wins 

Ties 

Losses 

A.c cults, cy IKM v ersus K-me ans 

'7 

4 

1 

I KM1 ver: sms. 13 e nsity 

9 

O 

3 

IKM -v ersus FF 

S 

O 

4 

IKM v ersus EM 

<5 

O 

<5 

IKM v ersus Hier 

S 

O 

4 

ADC I KMI ver s u s K-m e ans 

12 

O 

Q 

IKM v ersus 13 ensity 

9 

0 

3 

IKM v ersms FF 

1 1 

O 

1 

IKM v ersus EM 

6 

O 

6 

IKM v ersms Hier 

12 

O 

0 

Pre cision IKM v ersms K-me ans 

4 

O 

S 

IKM! v ersus 13 ensity 

2 

O 

10 

IKM v ersns FF 

S 

O 

4 

IKM v ersns EM 

3 

O 

9 

IKM v ersns Hier 

9 

0 

3 

Re c all IKM! v ersns K-me ans 

6 

0 

6 

I KM ver sn s 13 e nsity 

5 

O 

7 

IKM v ersns FF 

■6 

O 

6 

IKM v ersns EM 

5 

O 

7 

IKM v ersns Hier 

3 

O 

9 

F-me a sure IKM v ersns K-me ans 

5 

O 

7 

IKM v ersns 13 ensity 

5 

O 

7 

IKM v ersns FF 

S 

O 

4 

I KM v ersns EM 

4 

0 

S 

IKM v ersns Hier 

7 

O 

5 


Finally, we can say that IKM are one of the best alternatives to 
handle class imbalance problems effectively.This 
experimental study supports the conclusion that the a 
prominent recursive oversampling approach can improve the 
CIL behavior when dealing with imbalanced data-sets, as it 
has helped the IKM methods to be the best 
performingalgorithms when compared with four classical and 
well-known algorithms: K-means, Density, FF, EM and a 
well-established Hierarchical algorithm. 


VIII. Conclusion 

In this paper, a novel clustering algorithm for imbalanced 
distributed data hasbeen proposed. This method uses unique 
oversampling technique to almost balance dataset such that to 
minimize the “uniform effect 44 in the clustering process. 
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Empirical results have shown that IKM considerably reduces 
the uniform effect while retaining or improving the clustering 
measure when compared with benchmark methods. In fact, 
the proposedmethod may also be useful as a frame work for 
data sources for better clustering measures. 
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